{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "98a63a7e",
   "metadata": {},
   "source": [
    "# 推荐系统基础"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "156b35f0",
   "metadata": {},
   "source": [
    "Q1：推荐系统与常见的结构化问题的区别是什么？<br>\n",
    "A1：推荐系统需要各模块的协助来运行。当中有需要**用户画像**和**内容画像**等显性结构化数据，也需要参杂一下其他的非结构化隐形数据\n",
    "\n",
    "Q2: 如何评价推荐系统「推荐」的准不准？<br>\n",
    "A2：推荐系统可通过**硬指标**（如：点击的概率）、**软指标**（如：优秀内容推荐）来作为算法的衡量指标。而推荐的效果则可以通过**离线测试**、**用户反馈**、**在线测试**得到\n",
    "\n",
    "Q3：推荐系统一般分为召回 & 排序，为什么这样划分？<br>\n",
    "A3：因为一个系统中设计数据量巨大，为了降低对硬件资源的消耗。推荐系统需要先对数据进行召回，进而过滤大量的无用数据；然后然后再排序出用户感兴趣的呢日哦那个。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65698fa8",
   "metadata": {},
   "source": [
    "# Movienles介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "22b5a882",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "movie_df = pd.read_table(\"data/movies.dat\",sep = '::',header=None,engine='python')\n",
    "movie_df.columns = (['MovieID','Title','Genres'])\n",
    "\n",
    "user_df = pd.read_table(\"data/users.dat\",sep = '::',header=None,engine='python')\n",
    "user_df.columns = (['UserID','Gender','Age','Occupation','Zip-code'])\n",
    "\n",
    "rating_df = pd.read_table(\"data/ratings.dat\",sep = '::',header=None,engine='python')\n",
    "rating_df.columns = (['UserID','MovieID','Rating','Timestamp'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6cace68b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "一共有6040名用户\n",
      "一共有3883部电影\n"
     ]
    }
   ],
   "source": [
    "print('一共有{}名用户'.format(len(user_df['UserID'].unique())))\n",
    "print('一共有{}部电影'.format(len(movie_df['MovieID'].unique())))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "feeebfd1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "每个用户参与评分电影数\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "UserID\n",
       "1        53\n",
       "2       129\n",
       "3        51\n",
       "4        21\n",
       "5       198\n",
       "       ... \n",
       "6036    888\n",
       "6037    202\n",
       "6038     20\n",
       "6039    123\n",
       "6040    341\n",
       "Name: MovieID, Length: 6040, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('每个用户参与评分电影数')\n",
    "rating_df.groupby('UserID')['MovieID'].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "32a8d83f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "每部电影的平均得分\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "MovieID\n",
       "1       4.146846\n",
       "2       3.201141\n",
       "3       3.016736\n",
       "4       2.729412\n",
       "5       3.006757\n",
       "          ...   \n",
       "3948    3.635731\n",
       "3949    4.115132\n",
       "3950    3.666667\n",
       "3951    3.900000\n",
       "3952    3.780928\n",
       "Name: Rating, Length: 3706, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('每部电影的平均得分')\n",
    "rating_df.groupby('MovieID')['Rating'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "88399fba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "每个用户的平均评分\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "UserID\n",
       "1       4.188679\n",
       "2       3.713178\n",
       "3       3.901961\n",
       "4       4.190476\n",
       "5       3.146465\n",
       "          ...   \n",
       "6036    3.302928\n",
       "6037    3.717822\n",
       "6038    3.800000\n",
       "6039    3.878049\n",
       "6040    3.577713\n",
       "Name: Rating, Length: 6040, dtype: float64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('每个用户的平均评分')\n",
    "rating_df.groupby('UserID')['Rating'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9964a104",
   "metadata": {},
   "source": [
    "# 协同过滤基础"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a6b70f0b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# 定义数据集， 也就是那个表格， 注意这里我们采用字典存放数据， 因为实际情况中数据是非常稀疏的， 很少有情况是现在这样\n",
    "def loadData():\n",
    "    items={'A': {1: 5, 2: 3, 3: 4, 4: 3, 5: 1},\n",
    "           'B': {1: 3, 2: 1, 3: 3, 4: 3, 5: 5},\n",
    "           'C': {1: 4, 2: 2, 3: 4, 4: 1, 5: 5},\n",
    "           'D': {1: 4, 2: 3, 3: 3, 4: 5, 5: 2},\n",
    "           'E': {2: 3, 3: 5, 4: 4, 5: 1}\n",
    "          }\n",
    "    users={1: {'A': 5, 'B': 3, 'C': 4, 'D': 4},\n",
    "           2: {'A': 3, 'B': 1, 'C': 2, 'D': 3, 'E': 3},\n",
    "           3: {'A': 4, 'B': 3, 'C': 4, 'D': 3, 'E': 5},\n",
    "           4: {'A': 3, 'B': 3, 'C': 1, 'D': 5, 'E': 4},\n",
    "           5: {'A': 1, 'B': 5, 'C': 5, 'D': 2, 'E': 1}\n",
    "          }\n",
    "    return items,users\n",
    "\n",
    "items, users = loadData()\n",
    "item_df = pd.DataFrame(items).T\n",
    "user_df = pd.DataFrame(users).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "14a41745",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.852803</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.792118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.852803</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.467707</td>\n",
       "      <td>0.489956</td>\n",
       "      <td>-0.900149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.467707</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.161165</td>\n",
       "      <td>-0.466569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.489956</td>\n",
       "      <td>-0.161165</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.641503</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.792118</td>\n",
       "      <td>-0.900149</td>\n",
       "      <td>-0.466569</td>\n",
       "      <td>-0.641503</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          1         2         3         4         5\n",
       "1  0.000000  0.852803  0.707107  0.000000 -0.792118\n",
       "2  0.852803  0.000000  0.467707  0.489956 -0.900149\n",
       "3  0.707107  0.467707  0.000000 -0.161165 -0.466569\n",
       "4  0.000000  0.489956 -0.161165  0.000000 -0.641503\n",
       "5 -0.792118 -0.900149 -0.466569 -0.641503  0.000000"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"计算用户相似性矩阵\"\"\"\n",
    "similarity_matrix = pd.DataFrame(np.zeros((len(users), len(users))), index=[1, 2, 3, 4, 5], columns=[1, 2, 3, 4, 5])\n",
    "\n",
    "# 遍历每条用户-物品评分数据\n",
    "for userID in users:\n",
    "    for otheruserId in users:\n",
    "        vec_user = []\n",
    "        vec_otheruser = []\n",
    "        if userID != otheruserId:\n",
    "            for itemId in items:   # 遍历物品-用户评分数据\n",
    "                itemRatings = items[itemId]        # 这也是个字典  每条数据为所有用户对当前物品的评分\n",
    "                if userID in itemRatings and otheruserId in itemRatings:  # 说明两个用户都对该物品评过分\n",
    "                    vec_user.append(itemRatings[userID])\n",
    "                    vec_otheruser.append(itemRatings[otheruserId])\n",
    "            # 这里可以获得相似性矩阵(共现矩阵)\n",
    "            similarity_matrix[userID][otheruserId] = np.corrcoef(np.array(vec_user), np.array(vec_otheruser))[0][1]\n",
    "            #similarity_matrix[userID][otheruserId] = cosine_similarity(np.array(vec_user), np.array(vec_otheruser))[0][1]\n",
    "similarity_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "50acf312",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.476731</td>\n",
       "      <td>-0.123091</td>\n",
       "      <td>0.532181</td>\n",
       "      <td>0.969458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>-0.476731</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.645497</td>\n",
       "      <td>-0.310087</td>\n",
       "      <td>-0.478091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>C</th>\n",
       "      <td>-0.123091</td>\n",
       "      <td>0.645497</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.720577</td>\n",
       "      <td>-0.427618</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>D</th>\n",
       "      <td>0.532181</td>\n",
       "      <td>-0.310087</td>\n",
       "      <td>-0.720577</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.581675</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>E</th>\n",
       "      <td>0.969458</td>\n",
       "      <td>-0.478091</td>\n",
       "      <td>-0.427618</td>\n",
       "      <td>0.581675</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A         B         C         D         E\n",
       "A  1.000000 -0.476731 -0.123091  0.532181  0.969458\n",
       "B -0.476731  1.000000  0.645497 -0.310087 -0.478091\n",
       "C -0.123091  0.645497  1.000000 -0.720577 -0.427618\n",
       "D  0.532181 -0.310087 -0.720577  1.000000  0.581675\n",
       "E  0.969458 -0.478091 -0.427618  0.581675  1.000000"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"计算物品的相似矩阵\"\"\"\n",
    "similarity_matrix = pd.DataFrame(np.ones((len(items), len(items))), index=['A', 'B', 'C', 'D', 'E'], columns=['A', 'B', 'C', 'D', 'E'])\n",
    "\n",
    "# 遍历每条物品-用户评分数据\n",
    "for itemId in items:\n",
    "    for otheritemId in items:\n",
    "        vec_item = []         # 定义列表， 保存当前两个物品的向量值\n",
    "        vec_otheritem = []\n",
    "        #userRagingPairCount = 0     # 两件物品均评过分的用户数\n",
    "        if itemId != otheritemId:    # 物品不同\n",
    "            for userId in users:    # 遍历用户-物品评分数据\n",
    "                userRatings = users[userId]    # 每条数据为该用户对所有物品的评分， 这也是个字典\n",
    "                \n",
    "                if itemId in userRatings and otheritemId in userRatings:   # 用户对这两个物品都评过分\n",
    "                    #userRagingPairCount += 1\n",
    "                    vec_item.append(userRatings[itemId])\n",
    "                    vec_otheritem.append(userRatings[otheritemId])\n",
    "            \n",
    "            # 这里可以获得相似性矩阵(共现矩阵)\n",
    "            similarity_matrix[itemId][otheritemId] = np.corrcoef(np.array(vec_item), np.array(vec_otheritem))[0][1]\n",
    "            #similarity_matrix[itemId][otheritemId] = cosine_similarity(np.array(vec_item), np.array(vec_otheritem))[0][1]\n",
    "similarity_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f43a0a56",
   "metadata": {},
   "source": [
    "# 协同过滤进阶"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9fee188",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
